{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 在 Llama Index 中使用千帆能力\n",
    "\n",
    "LlamaIndex是一个用于连接大语言模型（LLMs）和外部数据源的数据框架。它能够让LLMs访问和利用私有或领域特定的数据，以优化模型性能并使其更加易用和流畅。\n",
    "\n",
    "本文准备了一份在 Llama Index 中使用千帆的能力进行 RAG 的示例，供用户参考。\n",
    "\n",
    "# 前置准备\n",
    "\n",
    "用户首先需要自行安装集成了 OpenAI 组件的 Llama Index，以及安装最新版本的千帆 Python SDK"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "vscode": {
     "languageId": "shellscript"
    }
   },
   "outputs": [],
   "source": [
    "!pip install -U \"qianfan[openai]\"\n",
    "!pip install -U llama-index\n",
    "!pip install -U nest_asyncio"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "在安装完成之后，我们还需要在 Shell 中设置相关环境变量并启动千帆的 OpenAI 转接服务。\n",
    "\n",
    "这个服务负责监听本地发出的、指向 OpenAI 的请求，并将请求映射并转发到千帆，最后从千帆拿回结果。\n",
    "\n",
    "从而实现使得各类适配了 OpenAI 服务的框架，能够无感、零代码接入千帆平台。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "OpenAI wrapper server is running at                                             \n",
      "\n",
      "\u001b[1;33m • \u001b[0mhttp://127.0.0.1:8001                                                        \n",
      "\u001b[1;33m • \u001b[0mhttp://172.18.183.188:8001                                                   \n",
      "\n",
      "Remember to set the environment variables:                                      \n",
      "\n",
      "\u001b[40m                                                                                \u001b[0m\n",
      "\u001b[40m \u001b[0m\u001b[97;40m    \u001b[0m\u001b[97;40mexport\u001b[0m\u001b[97;40m \u001b[0m\u001b[97;40mOPENAI_API_KEY\u001b[0m\u001b[91;40m=\u001b[0m\u001b[93;40m'any-content-you-want'\u001b[0m\u001b[40m                              \u001b[0m\u001b[40m \u001b[0m\n",
      "\u001b[40m \u001b[0m\u001b[97;40m    \u001b[0m\u001b[97;40mexport\u001b[0m\u001b[97;40m \u001b[0m\u001b[97;40mOPENAI_BASE_URL\u001b[0m\u001b[91;40m=\u001b[0m\u001b[93;40m'http://172.18.183.188:8001/v1'\u001b[0m\u001b[40m                    \u001b[0m\u001b[40m \u001b[0m\n",
      "\u001b[40m                                                                                \u001b[0m\n",
      "\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Started server process [44989]\n",
      "INFO:     Waiting for application startup.\n",
      "INFO:     Application startup complete.\n",
      "INFO:     Uvicorn running on http://0.0.0.0:8001 (Press CTRL+C to quit)\n"
     ]
    }
   ],
   "source": [
    "import nest_asyncio\n",
    "import os\n",
    "import subprocess as sp\n",
    "\n",
    "nest_asyncio.apply()\n",
    "\n",
    "# 该环境变量用于绕过 LlamaIndex 对 OpenAI API Key 的检查\n",
    "os.environ[\"OPENAI_API_KEY\"] = \"FAKE_KEY\"\n",
    "\n",
    "# 需要将 OPENAI 的 API 请求重定向到本地的服务\n",
    "os.environ[\"OPENAI_API_BASE\"] = \"http://127.0.0.1:8001/v1\"\n",
    "\n",
    "# 以下环境变量供千帆 OpenAI Adapter 使用\n",
    "os.environ[\"QIANFAN_ACCESS_KEY\"] = \"your_qianfan_console_access_key\"\n",
    "os.environ[\"QIANFAN_SECRET_KEY\"] = \"your_qianfan_console_secret_key\"\n",
    "os.environ[\"QIANFAN_QPS_LIMIT\"] = \"5\"\n",
    "\n",
    "# 启动千帆 SDK 的 OpenAI Adapter 服务，用于代理转发指向 OpenAI 的请求，并替换为千帆服务的返回\n",
    "server = sp.Popen(\"qianfan openai -p 8001\", shell=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 加载数据与构建向量库\n",
    "\n",
    "Llama Index 提供的数据加载功能允许我们直接从文件夹方便的构建向量数据库"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     127.0.0.1:58923 - \"POST /v1/embeddings HTTP/1.1\" 200 OK\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "\n",
      "Generating embeddings: 0it [00:00, ?it/s]\n"
     ]
    }
   ],
   "source": [
    "from llama_index.core import VectorStoreIndex, SimpleDirectoryReader\n",
    "\n",
    "documents = SimpleDirectoryReader(\"data\").load_data()\n",
    "\n",
    "from llama_index.embeddings.openai import OpenAIEmbedding\n",
    "from llama_index.core.node_parser import SentenceSplitter\n",
    "\n",
    "transformations=[\n",
    "    SentenceSplitter(chunk_size=50, chunk_overlap=0),\n",
    "    # 这里虽然使用的是 OpenAI 的 Embedding 服务组件\n",
    "    # 但是实际上发出去的请求会被重定向到千帆\n",
    "    # 由千帆进行实际的处理操作\n",
    "    # 目前实际使用的千帆模型为 Embeddings-V1\n",
    "    # 且只支持使用 Embeddings-V1 进行 Embedding\n",
    "    OpenAIEmbedding(embed_batch_size=10),\n",
    "]\n",
    "\n",
    "index = VectorStoreIndex.from_documents(documents, transformations=transformations, show_progress=True)\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# 查询文本\n",
    "\n",
    "在构建完成后，`index` 对象就是一个包含了上面文本的向量化表示的向量数据库。我们可以直接使用该向量数据库所提供的大模型查询方法 `query` ，用自然语言来查询向量数据库中的文本。"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [03-28 18:50:29] openapi_requestor.py:377 [t:8094817088]: async requesting llm api endpoint: /embeddings/embedding-v1\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     127.0.0.1:59061 - \"POST /v1/embeddings HTTP/1.1\" 200 OK\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[INFO] [03-28 18:50:30] openapi_requestor.py:377 [t:8094817088]: async requesting llm api endpoint: /chat/completions\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "INFO:     127.0.0.1:59063 - \"POST /v1/chat/completions HTTP/1.1\" 200 OK\n",
      "\n",
      "大模型回答: Based on the provided context information, there is no mention of what the author did growing up. The context only provides information about a file path and excerpts from a text file named \"paul_graham_essay.txt\". Therefore, it is not possible to answer the query based on the given context.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "# 这里支持传入OPEN AI 的模型名。相关的模型名会被映射到千帆的模型名。\n",
    "query_engine = index.as_query_engine()\n",
    "\n",
    "# 如果用户安装了 Langchain，还可以直接使用千帆在 Langchain 中的 ChatModel 组件\n",
    "# 来在 LlamaIndex 中直接使用千帆的大模型服务\n",
    "# 如下所示，使用前请解除注释\n",
    "# from langchain.chat_models.baidu_qianfan_endpoint import QianfanChatEndpoint\n",
    "# query_engine = index.as_chat_engine(llm=QianfanChatEndpoint())\n",
    "\n",
    "response = query_engine.query(\"What did the author do growing up?\")\n",
    "print(f\"\\n大模型回答: {response}\\n\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "最后关闭我们启动的服务"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "INFO:     Shutting down\n",
      "INFO:     Waiting for application shutdown.\n",
      "INFO:     Application shutdown complete.\n",
      "INFO:     Finished server process [44989]\n"
     ]
    },
    {
     "ename": "",
     "evalue": "",
     "output_type": "error",
     "traceback": [
      "\u001b[1;31m在当前单元格或上一个单元格中执行代码时 Kernel 崩溃。\n",
      "\u001b[1;31m请查看单元格中的代码，以确定故障的可能原因。\n",
      "\u001b[1;31m单击<a href='https://aka.ms/vscodeJupyterKernelCrash'>此处</a>了解详细信息。\n",
      "\u001b[1;31m有关更多详细信息，请查看 Jupyter <a href='command:jupyter.viewOutput'>log</a>。"
     ]
    }
   ],
   "source": [
    "server.terminate()"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "bce-qianfan-sdk-new",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}
